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What is l-O? 

Industrial-organizational (l-O) 
psychology is the scientific study of 
the workplace. Rigor and methods of 
psychology are applied to issues of 
critical relevance to business, 
including talent management, 
coaching, assessment, selection, 
training, organizational development, 
performance, and work-life balance. 
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In May, the Department of Education's Office of Civil Rights (OCR) released a draft Resource 
Discrimination in High Stakes Testing" that sought to provide an overview of federal standards 
principles that should guide the use of tests for making high stakes educational decisions (e.c 
special educational referrals, promotion, graduation, and scholarship awards). This Resource 
development for several years according to OCR, but educational groups and test publishers 
working days before it was originally scheduled for release. 

The Guide may have limited direct impact on 1-0 psychologists, unless they are involved in ed 
However, the Guide may be of interest for other reasons, since it interprets and applies both I 
employment arena and professional testing standards to issues of disparate impact in ways tl 
"overreaching" or incorrect. 

Test publishers, APA, and other educational institutions objected to the proposed timing of the 
agreed to revise the current document with plans for a fall publication. OCR has stated the Gi 
new federal guidelines or professional standards, but rather will provide a meaningful interpre 
tests in education. A number of national media outlets (New York Times , Wall Street Journal, 
Chronicle of Higher Education ) have run stories on the guidelines and op-ed pieces that have 
emphasis on disparate impact being the sole determination of whether or not a test should be 

The Guide cites specific wording from the Standards for Educational and Psychological Testir 
40 occasions, leading APA, AERA, and NOME to formally request that OCR delay revision of 
has been revised and published (sometime around December 1999). Several organizations h 
comments on the OCR Guide. 
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The Guide attempts to apply Title VII law, EEOC Guidelines, and professional standards that ; 
to educational test use. It cites several Supreme Court and lower courts decisions concerning 
or transports decisions and standards to education. Major concerns addressed by education 
summarized in comments submitted by the College Board (Camara, 6/21/1999): 


First, the Resource Guide focuses exclusively on disparate impact resulting from tests (or diff 
ignores the level of validity and utility offered by a test. Disparate impact cannot be considere 
must be evaluated in terms of the overall validity and utility of inferences associated with the | 
Resource Guide clearly elevates any measure, irrespective of validity, cost, or burden to the ( 
lower disparate outcomes above any test having greater disparate outcomes. We believe this 
precedent that has no legal or professional justification and the Guide will have a chilling effei 
educational tests. 


Second, the Resource Guide offers no guidance on what level of disparate impact would rest 
there be substantial statistical disparities or would any disparate outcome result in an investig 
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should not be the primary statistical analysis used to determine if and when an alternative me 
A consistent pattern of ethnic and racial disparities has been found across a variety of standa 
National Assessment of Educational Progress (NAEP) and the National Educational Longitud 
educational measures used for high-stakes decisions, such as high school grades, class rank 
quality and rigor of courses completed, as well as educational outcomes (e.g., college grades 
(Camara and Schmidt, under review). Disparities in test results reflect similar differences in oi 
(e.g., job performance, college achievement, and grades) and may be indicative of earlier diff 
learn and educational opportunities, not test bias or flaws with the test. 

Third, professional and technical standards do not define tests so narrowly that they exclude 
assessments that are both used daily to make high-stakes decisions about individual student: 
have similar levels of disparate impact against protected groups. Specifically, the Test Stand* 
standardized ability (aptitude and achievement) instruments, diagnostic and evaluative device 
personality inventories, and projective instrumentsa more appropriate choice among assessn 
use will be facilitated if there is a reasonable comparability in the kinds of information availabl 
three broad categories of test instruments are covered [emphasis added]: constructed perfor 
and to a lesser extent, structured behavioral samples (pages, 3_4). M Related to this comment 
Resource Guide be renamed to put added emphasis on Measures Used in Making High-Stak 
Uniform Guidelines and Employment Selection Procedures ), rather than focus exclusively on 
decision-making process, testing. 

Fourth, we applaud OCR's deference to the Test Standards. However, the Resource Guide ir 
professional standards can be applied in a rigid manner in evaluating tests. The Test Standar 
rigid checklist approach, noting that specific circumstances affect the relevance of standards 
must be applied in evaluating tests. Professional practice and standards are typically construi 
other measures need not meet all standards to be appropriately used within the bounds of pri 
(Richardson, 729 F. Supp. At 821, 823). In addition, the three sponsoring educational associc 
the Standards, which date back to 1985. We strongly endorse the recommendations from AP 
asking that issuance of this Resource Guide be deferred until after publication and dissemina 
Standards and requesting a standard 90-day review period for any subsequent drafts of this c 
publication of the revised Test Standards. 

Fifth, we would ask OCR to ensure that colleges and universities, school districts, and state e 
an opportunity to review and comment on this proposed Resource Guide. The Resource Guic 
disseminated or reviewed by colleges and secondary schools. These are the very organizatic 
affected by the Resource Guide once it is issued and it seems appropriate that they be given 
comment on the inferences and proposed standards. 

Sixth, the distinction the Resource Guide makes between tests and other assessment device 
establishing a much lower technical, professional, and legal standard for more subjective ass 
applications, grades and GPA, recommendations, ratings or evaluations of student work and 
experiences and honors, community service and involvement, samples of student work). In 1/1 
and Trust , 487 U.S. 977, the American Psychological Association submitted an amicus curiae 
argued there is no professional or scientific justification to treat subjective and objective devic 
validation requirements. In fact, not imposing essentially the same legal and technical standa 
and devices used in high-stakes individual decisions would provide a sanctioned and covert r 
APA further argued that subjective procedures (in that case used for employment) are "amen 
psychometric scrutiny" as objective procedures, citing the Test Standards which address inte 
(Camara, 1996). In deciding Watson v. Fort Worth Bank and Trust , 487 U.S. 977, all eight of 
O'Connor's opinion holding that the adverse impact theory can be used in cases involving sul 
was concerned that an employer could combine an objective criteria (such as a test or diplorr 
(such as interviews or ratings) and easily insulate itself from the Griggs test. O'Connor noted 
making systems could have "precisely the same effects as a system pervaded by impermissit 
(Opinion at 4926). 

Seventh, professional and legal standards do not provide any support for OCR's distinctions I 
and other measures. We agree with comments to an earlier draft of this Resource Guide subi 
Testing and Assessment (Shavelson, June 10, 1996), stating that "OCR's inquiry is not to prc 
validity of inferences and decisions based on tests, but rather to determine whether the entire 
a part provides students a fair and equal opportunity to learn...." The Resource Guide ignores 
even if they contribute more to disparate outcomes. In fact, high school courses, judgments a 
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high school curriculum," grades, and rank may also contribute more to disparate outcomes, t\ 
testing, if an institution places substantially greater weight on these factors. For example, if te 
one of several factors in admissions, then there is no guarantee those disparate outcomes wi 
eliminated. In requiring tests to meet an exceptionally higher standard than other measures (< 
student work, high school rank, past experience, and opportunities), the Resource Guide will 
of valid and objective standardized tests used by educational institutions, states, and school c 
educational institutions may opt to employ less valid and less objective methods for high-stak 
are not addressed in this Resource Guide. 

Eighth, the Resource Guide also sanctions the use of the Uniform Guidelines on Employmen 
resource in educational testing. As the Resource Guide acknowledges in a footnote, there an 
differences between educational and employment testing that we believe undermines any att 
in educational settings. The Uniform Guidelines were never developed with application to edi 
organizations did not have an opportunity to comment on extensions of the principles to educ 
Guidelines are over 25 years old and do not reflect current scientific principles of measureme 
practice. The Uniform Guidelines are outdated and do not conform to the Testing Standards ( 
their consideration of validity (as accomplished by adopting one of three distinct types of valic 
(this is virtually ignored in the Guidelines , but is accepted professional practice), differential p 
as well as several other areas (APA, 1985). The Uniform Guidelines may provide a framewor 
guidelines addressing test use, but they should not be viewed as a substantive resource in ec 

Ninth, statistical analyses should be based on the pool of qualified applicants, not a general p 
not addressed in the Resource Guide. 

Tenth, this Resource Guide implies that once disparate impact is established that the burden 
educational institution to demonstrate both the educational necessity of the test and then to d 
alternative exists throughout the process. This legal interpretation is incorrect. 

Other sections of the Resource Guide viewed as problematic include wording implying that se 
studies are required for each school; that tests can only be used for purposes they were origi 
than for uses where sufficient validation evidence exist); and that there is a unique methodolc 
when they are to be used as the sole criteria. 

On June 18 th , the House held a hearing on the OCR Guide and department officials noted tha 
recirculate the current draft to groups who have already submitted comments on the current c 
submit a revised Guide to the National Academy of Sciences Board of Testing and Assessme 
Thereafter, they anticipate making a final draft available for public review this fall. They will pi 
and will have the revised Guide posted on their web (Coleman, June 21, 1999, personal com 
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